<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "https://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=11"/>
<meta name="generator" content="Doxygen 1.9.4"/>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<title>Flow-IPC: Shared Memory: Direct Allocation, Transport</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="search/searchdata.js"></script>
<script type="text/javascript" src="search/search.js"></script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
 <tbody>
 <tr id="projectrow">
  <td id="projectalign">
   <div id="projectname">Flow-IPC<span id="projectnumber">&#160;1.0.2</span>
   </div>
   <div id="projectbrief">Flow-IPC project: Full implementation reference.</div>
  </td>
 </tr>
 </tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.9.4 -->
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:d3d9a9a6595521f9666a5e94cc830dab83b65699&amp;dn=expat.txt MIT */
var searchBox = new SearchBox("searchBox", "search",'Search','.html');
/* @license-end */
</script>
<script type="text/javascript" src="menudata.js"></script>
<script type="text/javascript" src="menu.js"></script>
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:d3d9a9a6595521f9666a5e94cc830dab83b65699&amp;dn=expat.txt MIT */
$(function() {
  initMenu('',true,false,'search.php','Search');
  $(document).ready(function() { init_search(); });
});
/* @license-end */
</script>
<div id="main-nav"></div>
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
     onmouseover="return searchBox.OnSearchSelectShow()"
     onmouseout="return searchBox.OnSearchSelectHide()"
     onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>

<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0" 
        name="MSearchResults" id="MSearchResults">
</iframe>
</div>

</div><!-- top -->
<div><div class="header">
  <div class="headertitle"><div class="title">Shared Memory: Direct Allocation, Transport </div></div>
</div><!--header-->
<div class="contents">
<div class="textblock"><center><b>MANUAL NAVIGATION:</b> <a class="el" href="chan_struct_advanced.html">Preceding Page</a> - <a class="el" href="transport_core.html">Next Page</a> - <a href="./pages.html"><b>Table of Contents</b></a> - <a class="el" href="namespaceipc.html"><b>Reference</b></a></center><hr  />
<p >Here we discuss the direct use of SHM arenas for allocation of C++ data structures; and how to share such structures between processes via <a class="el" href="namespaceipc_1_1transport.html" title="Flow-IPC module providing transmission of structured messages and/or low-level blobs (and more) betwe...">ipc::transport</a>. (Or go back to the prerequisite preceding page: <a class="el" href="chan_struct_advanced.html">Structured Message Transport: Messages As Long-lived Data Structures / Advanced Topics</a>.)</p>
<h2>How shared memory (SHM) fits into Flow-IPC </h2>
<p >If you've been reading this Manual sequentially to this point, you've seen <b>shared memory (SHM)</b> mentioned many times already. Nevertheless our philosophy on how much the user &ndash; you &ndash; should have to think about SHM is as follows.</p>
<p >We don't want you to think about it any more than absolutely necessary. It is a means to an end: generally speaking avoiding copying data shared among processes, for performance and algorithmic ease. As such much of the work with Flow-IPC should proceed without code mentioning SHM at all. Case in point: <code>struc::Channel</code> allows one to transmit messages with zero-copy performance, which means SHM is <em>internally</em> used, but you, the <em>user</em>, get that benefit automatically: All you have to do, essentially, is say in code, "I want zero-copy performance." To do that, ultimately, all you do is (on each side): when setting up your IPC session(s), specify a certain <code>Session</code> or <code>Session_server</code> base type which will enable SHM-backed performance with no further references to that fact in the rest of your code. To recap the recipes given in <a class="el" href="session_setup.html">Sessions: Setting Up an IPC Context</a> and <a class="el" href="chan_struct.html">Structured Message Transport</a> &ndash;</p>
<p ><a class="anchor" id="transport_shm_setup"></a>Session-client:</p>
<div class="fragment"><div class="line"><span class="keyword">template</span>&lt;ipc::session::schema::MqType S_MQ_TYPE_OR_NONE, <span class="keywordtype">bool</span> S_TRANSMIT_NATIVE_HANDLES,</div>
<div class="line">         <span class="keyword">typename</span> Mdt_payload = ::capnp::Void&gt;</div>
<div class="line"><span class="comment">// Right here --------------------------v-- is where zero-copy (SHM-backed) engine is specified.</span></div>
<div class="line"><span class="keyword">using </span>Client_session_t = <a class="code hl_class" href="classipc_1_1session_1_1shm_1_1classic_1_1Session__mv.html">ipc::session::shm::classic::Client_session&lt;S_MQ_TYPE_OR_NONE, S_TRANSMIT_NATIVE_HANDLES, Mdt_payload&gt;</a>;</div>
<div class="line"><span class="comment">// Alternatively you could have written the following --v-- to specify a different SHM-backed engine.</span></div>
<div class="line"><span class="keyword">using </span>Client_session_t = <a class="code hl_class" href="classipc_1_1session_1_1shm_1_1arena__lend_1_1jemalloc_1_1Session__mv.html">ipc::session::shm::arena_lend::jemalloc::Client_session&lt;S_MQ_TYPE_OR_NONE, S_TRANSMIT_NATIVE_HANDLES, Mdt_payload&gt;</a>;</div>
<div class="line"><span class="comment">// Or if you don&#39;t want zero copy: --v</span></div>
<div class="line"><span class="keyword">using </span>Client_session_t = <a class="code hl_class" href="classipc_1_1session_1_1Client__session__mv.html">ipc::session::Client_session&lt;S_MQ_TYPE_OR_NONE, S_TRANSMIT_NATIVE_HANDLES, Mdt_payload&gt;</a>;</div>
<div class="line"> </div>
<div class="line"><span class="comment">// Now everything will flow from the above, alias to alias to alias... no SHM details being mentioned:</span></div>
<div class="line"><span class="keyword">using </span>Session = Client_session_t&lt;session::schema::MqType::NONE, true&gt;;</div>
<div class="line"><span class="keyword">template</span>&lt;<span class="keyword">typename</span> Message_body&gt;</div>
<div class="line"><span class="keyword">using </span>Structured_channel_t = Session::Structured_channel&lt;Message_body&gt;;</div>
<div class="line"> </div>
<div class="line"><span class="keyword">using </span>Cool_structured_channel = Structured_channel_t&lt;my_meta_app::capnp::CoolMsg&gt;;</div>
<div class="line"><span class="comment">// ...etc....</span></div>
<div class="ttc" id="aclassipc_1_1session_1_1Client__session__mv_html"><div class="ttname"><a href="classipc_1_1session_1_1Client__session__mv.html">ipc::session::Client_session_mv</a></div><div class="ttdoc">Implements Session concept on the Client_app end: a Session_mv that first achieves PEER state by conn...</div><div class="ttdef"><b>Definition:</b> <a href="ipc__session_2src_2ipc_2session_2client__session_8hpp_source.html#l00143">client_session.hpp:145</a></div></div>
<div class="ttc" id="aclassipc_1_1session_1_1shm_1_1arena__lend_1_1jemalloc_1_1Session__mv_html"><div class="ttname"><a href="classipc_1_1session_1_1shm_1_1arena__lend_1_1jemalloc_1_1Session__mv.html">ipc::session::shm::arena_lend::jemalloc::Session_mv</a></div><div class="ttdoc">Implements the SHM-related API common to shm::arena_lend::jemalloc::Server_session and shm::arena_len...</div><div class="ttdef"><b>Definition:</b> <a href="ipc__shm__arena__lend_2src_2ipc_2session_2shm_2arena__lend_2jemalloc_2session_8hpp_source.html#l00050">session.hpp:52</a></div></div>
<div class="ttc" id="aclassipc_1_1session_1_1shm_1_1classic_1_1Session__mv_html"><div class="ttname"><a href="classipc_1_1session_1_1shm_1_1classic_1_1Session__mv.html">ipc::session::shm::classic::Session_mv</a></div><div class="ttdoc">Implements the SHM-related API common to shm::classic::Server_session and shm::classic::Client_sessio...</div><div class="ttdef"><b>Definition:</b> <a href="ipc__shm_2src_2ipc_2session_2shm_2classic_2session_8hpp_source.html#l00042">session.hpp:44</a></div></div>
</div><!-- fragment --><p >Session-server:</p>
<div class="fragment"><div class="line"><span class="keyword">template</span>&lt;ipc::session::schema::MqType S_MQ_TYPE_OR_NONE, <span class="keywordtype">bool</span> S_TRANSMIT_NATIVE_HANDLES,</div>
<div class="line">         <span class="keyword">typename</span> Mdt_payload = ::capnp::Void&gt;</div>
<div class="line"><span class="keyword">using </span>Session_server_t = <a class="code hl_class" href="classipc_1_1session_1_1shm_1_1classic_1_1Session__server.html">ipc::session::shm::classic::Session_server&lt;S_MQ_TYPE_OR_NONE, S_TRANSMIT_NATIVE_HANDLES, Mdt_payload&gt;</a>;</div>
<div class="line"><span class="comment">// Alternatively (different SHM-backed engine):</span></div>
<div class="line"><span class="keyword">using </span>Session_server_t = <a class="code hl_class" href="classipc_1_1session_1_1shm_1_1arena__lend_1_1jemalloc_1_1Session__server.html">ipc::session::shm::arena_lend::jemalloc::Session_server&lt;S_MQ_TYPE_OR_NONE, S_TRANSMIT_NATIVE_HANDLES, Mdt_payload&gt;</a>;</div>
<div class="line"><span class="comment">// Alternatively (no zero-copy/SHM-backing):</span></div>
<div class="line"><span class="keyword">using </span>Session_server_t = ipc::session::shm::Session_server&lt;S_MQ_TYPE_OR_NONE, S_TRANSMIT_NATIVE_HANDLES, Mdt_payload&gt;;</div>
<div class="line"> </div>
<div class="line"><span class="comment">// Now everything will flow from the above, alias to alias to alias... no SHM details being mentioned:</span></div>
<div class="line"><span class="keyword">using </span>Session_server = Session_server_t&lt;session::schema::MqType::NONE, true&gt;;</div>
<div class="line"><span class="keyword">using </span>Session = Session_server::Server_session_obj;</div>
<div class="line"><span class="keyword">template</span>&lt;<span class="keyword">typename</span> Message_body&gt;</div>
<div class="line"><span class="keyword">using </span>Structured_channel_t = Session::Structured_channel&lt;Message_body&gt;;</div>
<div class="line"> </div>
<div class="line"><span class="keyword">using </span>Cool_structured_channel = Structured_channel_t&lt;my_meta_app::capnp::CoolMsg&gt;;</div>
<div class="line"><span class="comment">// ...etc....</span></div>
<div class="ttc" id="aclassipc_1_1session_1_1shm_1_1arena__lend_1_1jemalloc_1_1Session__server_html"><div class="ttname"><a href="classipc_1_1session_1_1shm_1_1arena__lend_1_1jemalloc_1_1Session__server.html">ipc::session::shm::arena_lend::jemalloc::Session_server</a></div><div class="ttdoc">This is to vanilla Session_server what shm::arena_lend::jemalloc::Server_session is to vanilla Server...</div><div class="ttdef"><b>Definition:</b> <a href="ipc__shm__arena__lend_2src_2ipc_2session_2shm_2arena__lend_2jemalloc_2session__server_8hpp_source.html#l00070">session_server.hpp:74</a></div></div>
<div class="ttc" id="aclassipc_1_1session_1_1shm_1_1classic_1_1Session__server_html"><div class="ttname"><a href="classipc_1_1session_1_1shm_1_1classic_1_1Session__server.html">ipc::session::shm::classic::Session_server</a></div><div class="ttdoc">This is to vanilla Session_server what shm::classic::Server_session is to vanilla Server_session: it ...</div><div class="ttdef"><b>Definition:</b> <a href="ipc__shm_2src_2ipc_2session_2shm_2classic_2session__server_8hpp_source.html#l00070">session_server.hpp:74</a></div></div>
</div><!-- fragment --><p >Then it just works. Now, naturally, when working with structured data in, e.g., our example <code>Cool_structured_channel</code>, we may want to think beyond mere messaging and treat messages as shared data structures, in which case you <em>will</em> need to think about shared memory somewhat &ndash; but in a limited way. We went over that in <a class="el" href="chan_struct_advanced.html">Structured Message Transport: Messages As Long-lived Data Structures / Advanced Topics</a>. Even there, though, SHM-related thinking is limited to algorithmic decisions: when is a given <code>Msg_out</code> in existence; when does its lifetime end; which side should write; how to prevent concurrent non-read-only access (synchronization). You still don't need to think about SHM pools or SHM arenas. Notably, annoying and difficult things involving SHM pool naming and cleanup, including in case of crash, are performed invisibly and without your participation.</p>
<p >As pointed out in <a class="el" href="chan_struct_advanced.html">Structured Message Transport: Messages As Long-lived Data Structures / Advanced Topics</a>, however, working on shared data structures living <em>directly and exclusively</em> inside capnp-generated trees may be insufficient for some use cases. To recap possible ways in which it may be insufficient:</p><ul>
<li>A capnp tree <em>can</em> express almost any data structure, as it supports various scalars, <code>struct</code>s, <code>union</code>s, and dynamic-length lists... <em>But</em>: high-performance structures such as sorted trees and hash tables would require at least an entire layer of code to conveniently treat a capnp-<code>struct</code> substrate in that fashion. Plus the way capnp allocates space inside its segments, when a data structure is modified is not space-efficient or necessarily fast: It's not even <em>trying</em> to replicate a high-performance heap allocation algorithm. (The simplest example is what happens when a <code>List</code> size increases: capnp does <em>not</em> as of this writing allow the old-sized list's space to be reused subsequently <em>at all</em>; yet it continues taking RAM.)</li>
<li>As of this writing a received message instance (<code>Msg_in</code>) is a read-only view of the original out-message (even if, as we recommend, you use SHM-backing, and therefore it's really "viewing" the original structure directly where it was originally written). (As noted in <a class="el" href="chan_struct_advanced.html">Structured Message Transport: Messages As Long-lived Data Structures / Advanced Topics</a> this could be changed in the foreseeable future; but not as of this writing.) What if you'd like the recipient to write to the data structure?</li>
</ul>
<p >All that said, perhaps the best way to look at it is not: what can't you do with <code>struc::Channel</code> message directly? Rather look at it affirmatively: When two threads in a monolithic process want to participate in an algorithm on data structure X, how is that written? Well, one thread or the other creates on the heap and writes it initially; then a pointer is passed to the other thread; and then both of them simply access X <em>collaboratively</em>: reading, writing, synchronizing via mutex if necessary; and so on. So <code>struc::Msg_out</code> and <code>struc::Msg_in</code> are all well and good, but what if you simply want two threads <em>in two different processes</em> to <em>collaborate</em> in working on a given C++ data structure?</p>
<p >In that case you will want to <em>use SHM directly</em>.</p>
<p >Flow-IPC provides this ability and goes significantly further than known other SHM-access libraries, including the delightful boost.interprocess, in hiding pain from you. You won't have to worry about creating SHM pools, cleanup, naming. Perhaps most notably you won't have to write your own STL allocators or else forego the use of STL-compliant data structures due to their default code making them unsuitable for use with SHM. Flow-IPC provides a workflow to enable STL-compliant containers to be used directly in SHM. Manual/intrusive data structures, such as home-grown linked lists, are also supported. (boost.interprocess tries to do all this too, but using it involves painful stateful allocators and various other limitations. Certainly it does not leverage a commercial-grade memory manager like jemalloc; but we do. This is not actually a criticism of boost.interprocess; what it provides is very useful, and we leverage it ourselves. Just its ambition is lesser than ours: we build on it. Even excluding SHM-jemalloc and staying with SHM-classic we provide a powerful layer of usability on top of boost.interprocess.)</p>
<h3><a class="el" href="namespaceipc_1_1session.html" title="Flow-IPC module providing the broad lifecycle and shared-resource organization – via the session conc...">ipc::session</a> and SHM</h3>
<p >As you'll soon see, to allocate something in SHM and then share it with another process (where it can further access that "something," including modifying it if desired), one needs a <b>SHM arena</b> in which to allocate in the first place. Where do you get this arena, and how long will it keep existing? Generally there are 2 mutually exclusive answers to this question:</p><ul>
<li>(With <a class="el" href="namespaceipc_1_1session.html" title="Flow-IPC module providing the broad lifecycle and shared-resource organization – via the session conc...">ipc::session</a>) You can get it through the <a class="el" href="namespaceipc_1_1session.html" title="Flow-IPC module providing the broad lifecycle and shared-resource organization – via the session conc...">ipc::session</a> paradigm. You start an <a class="el" href="classipc_1_1session_1_1Session.html" title="A documentation-only concept defining the local side of an IPC conversation (session) with another en...">ipc::session::Session</a> (<a class="el" href="session_setup.html">Sessions: Setting Up an IPC Context</a>); and that object has 2 SHM-arenas explicitly available; most notably <code>session.session_shm()</code> returns a pointer to a SHM-arena object. You can allocate via that, and off you go.</li>
<li>(Without <a class="el" href="namespaceipc_1_1session.html" title="Flow-IPC module providing the broad lifecycle and shared-resource organization – via the session conc...">ipc::session</a>) You can create it manually by using something in <a class="el" href="namespaceipc_1_1shm.html" title="Modules for SHared Memory (SHM) support.">ipc::shm</a>. E.g., you could manually construct an <a class="el" href="classipc_1_1shm_1_1classic_1_1Pool__arena.html" title="A SHM-classic interface around a single SHM pool with allocation-algorithm services by boost....">ipc::shm::classic::Pool_arena</a>.</li>
</ul>
<p >In this page we limit discussion to way 1 (with <a class="el" href="namespaceipc_1_1session.html" title="Flow-IPC module providing the broad lifecycle and shared-resource organization – via the session conc...">ipc::session</a>). For way 2: documentation is available in the Reference &ndash; see various items, starting at sub-namespaces, under <a class="el" href="namespaceipc_1_1shm.html" title="Modules for SHared Memory (SHM) support.">ipc::shm</a>. For example see <a class="el" href="classipc_1_1shm_1_1classic_1_1Pool__arena.html" title="A SHM-classic interface around a single SHM pool with allocation-algorithm services by boost....">ipc::shm::classic::Pool_arena</a> docs.</p>
<p ><a class="anchor" id="shm_choice"></a></p><h3>Choice of SHM-provider</h3>
<p >There are two as of this writing:</p><ul>
<li>SHM-classic: <a class="el" href="namespaceipc_1_1shm_1_1classic.html" title="ipc::shm sub-module with the SHM-classic SHM-provider. See ipc::shm doc header for introduction.">ipc::shm::classic</a> (&lt;=&gt; <a class="el" href="namespaceipc_1_1session_1_1shm_1_1classic.html" title="Support for SHM-backed ipc::session sessions and session-servers with the SHM-classic (ipc::shm::clas...">ipc::session::shm::classic</a>); and</li>
<li>SHM-jemalloc: ipc::shm::arena_lend::jemalloc (&lt;=&gt; <a class="el" href="namespaceipc_1_1session_1_1shm_1_1arena__lend_1_1jemalloc.html" title="Support for SHM-backed ipc::session sessions and session-servers with the SHM-jemalloc (ipc::shm::are...">ipc::session::shm::arena_lend::jemalloc</a>).</li>
</ul>
<p >The majority of the API is identical between them; much code could be written generically, with one being substituted for the other at compile-time at will. (In the code snippet above you can see where one would make the change, on either side.) SHM-classic does have a couple of capabilities that SHM-jemalloc lacks. To recap these here for convenience:</p><ul>
<li>A session-client process can create <a href="./session_app_org.html#scope">app-scope</a> objects. (In SHM-jemalloc only a session-server process can do so. This is a basic limitation of arena-lending SHM-providers and is unlikely to change in the foreseeable future.)</li>
<li>A receiving (borrowing) process &ndash; not just the original allocating process &ndash; can write to the data structure. (In SHM-jemalloc receiving-side writing is precluded at the kernel level. We could optionally disable this limitation, as internally it is nothing more than a flag to a sys-call. However this would lose valuable safety aspects that SHM-jemalloc holds over SHM-classic as an advantage for some applications. So it would be a trade-off. Nevertheless it's likely to become available in the foreseeable future.)</li>
</ul>
<p >Conversely SHM-jemalloc, while having a somewhat reduced API set (enumerated above), has some major advantages too. See a <a href="./safety_perms.html#shm_safety">safety-oriented discussion</a> and <a href="./safety_perms.html#shm_safety_other_considerations">nearby recap of non-safety considerations</a>. A lengthier description can be <a class="el" href="namespaceipc_1_1shm.html">found in the Reference</a>.</p>
<p >In further discussion we will be agnostic as to the chosen SHM-provider wherever possible, with provider-specific notes where there's a difference in behavior or capability.</p>
<h2>Direct use of SHM: How-to </h2>
<p >The first step is to select SHM-backed sessions at compile-time; everything flows from that decision. See <a class="el" href="transport_shm.html#transport_shm_setup">above</a> for recipe.</p>
<h3>The 1-2 arenas available upon session-establishment</h3>
<p >SHM-backed capabilities become available, like everything else in the <a class="el" href="namespaceipc_1_1session.html" title="Flow-IPC module providing the broad lifecycle and shared-resource organization – via the session conc...">ipc::session</a> paradigm, once a session has been established in PEER state (<a class="el" href="session_setup.html">Sessions: Setting Up an IPC Context</a>). Internally during the session-opening procedure the following occurs. (It occurs invisibly, and without <a class="el" href="namespaceipc_1_1session.html" title="Flow-IPC module providing the broad lifecycle and shared-resource organization – via the session conc...">ipc::session</a> you'd have to create relevant arena(s) yourself which is very different between the 2 SHM-providers and much more difficult with SHM-jemalloc, though even with SHM-classic very annoying naming decisions would need to be made; not to mention subsequent cleanup headaches.)</p><ul>
<li>A <b>session-scope arena</b> (a/k/a <code>.session_shm()</code>) is created. This arena's <b>lifetime</b> (and therefore the max lifetime of any objects allocated by either side within it) is <em>until the end of the session</em>.</li>
<li>If and only if this has not yet occurred earlier for the distinct <a class="el" href="structipc_1_1session_1_1Client__app.html" title="An App that is used as a client in at least one client-server IPC split.">ipc::session::Client_app</a> involved in this session:<ul>
<li>An <b>app-scope arena</b> (a/k/a <code>.app_shm()</code>), for that specific distinct <code>Client_app</code>, is created. This arena's <b>lifetime</b> is <em>until the <code>Session_server</code> is destroyed</em>. That is: its lifetime spans all future sessions, not just the one that triggered its creation (in on-demand fashion).</li>
</ul>
</li>
<li>If the app-scope arena has already been created, it is not re-created; but it is made equally accessible via <code>.app_shm()</code> accessors.</li>
</ul>
<dl class="section note"><dt>Note</dt><dd>With SHM-jemalloc the app-scope arena is accessible (can be allocated in) only on the session-server end. The <code>.app_shm()</code> accesors do not exist on the session-client end.</dd></dl>
<hr  />
<dl class="section user"><dt>SHM-arena capacity</dt><dd>How much "stuff" is it possible to allocate in each arena? The answer is: it is essentially unlimited.</dd></dl>
<dl class="section user"><dt></dt><dd>More specifically, with SHM-jemalloc it is simply unlimited for all practical purposes.</dd></dl>
<dl class="section user"><dt></dt><dd>With SHM-classic there is as of this writing a hard-coded limit in the <em>gigabytes</em> (query <a class="el" href="classipc_1_1session_1_1shm_1_1classic_1_1Session__server.html#af6472e74db4dee7cf5ce5be55ded4179" title="The pool-size value, in mebibytes, which will be used to size the pool in subsequent async_accept()s.">ipc::session::shm::classic::Session_server::pool_size_limit_mi()</a> to get the value). If it proves insufficient, you can increase it via the similarly-named mutator <code>pool_size_limit_mi()</code>. Rest assured that all these gigabytes are <em>not</em> taken-away from general OS use from the get-go: Rather a given <em>page</em> (default size 4Ki) is taken-away from general use, only once it is "touched" by an allocation or write.</dd></dl>
<dl class="params"><dt>Parameters</dt><dd>
  <table class="params">
    <tr><td class="paramname">Unfortunately,however,at</td><td>least in Linux there are some kernel parameters governing how much virtual SHM space can be reserved in such a way, even though ~no physical RAM is taken by just creating a pool sized in the gigabytes. In particular if system-wide active pools exceed a certain parameter <code>Session_server::async_accept()</code> can yield "No space left on device" (<code>ENOSPC</code>) error. In this case one can: (1) tweak the kernel parameter(s) as admin; (2) reduce a given <code>Session_server</code>s' pool-size via the aforementioned mutator; or (3) use SHM-jemalloc provider which adjusts dynamically and creates/destroys smaller pools internally as needed.</td></tr>
  </table>
  </dd>
</dl>
<hr  />
<h3>Allocating in an arena</h3>
<p >Before something can be put into SHM and subsequently shared, one must allocate in a SHM arena. This idea is no different from how the regular heap is used, obviously. However instead of saying "allocate in the heap," one says "allocate in specific arena X."</p>
<p >First step is specify the arena. To do so use a <code>.session_shm()</code> or <code>.app_shm()</code> accessor; in all cases it returns a pointer to an arena object.</p>
<div class="fragment"><div class="line"><span class="keyword">auto</span> session_shm = session.session_shm(); <span class="comment">// session_shm is a pointer to an arena object.  *session_shm is the arena exclusively associated with `session`.</span></div>
<div class="line"><span class="keyword">auto</span> app_shm = session.app_shm(); <span class="comment">// .app_shm() is the arena shared among all sessions with the same Client_app (by `.m_name`) as `session`.</span></div>
<div class="line"><span class="comment">// Alternative way to access this, in case a specific `session` is not easily available.  Returns null, if no session with Client_app a_client_app has yet opened.</span></div>
<div class="line"><span class="keyword">auto</span> app_shm = session_server.app_shm(a_client_app);</div>
<div class="line"> </div>
<div class="line"><span class="keyword">auto</span> cool_obj = session_shm.construct&lt;Cool_obj&gt;(...); <span class="comment">// And off we go allocating in an arena.</span></div>
</div><!-- fragment --><p >Second step is to construct something within it. While certain lower-level capabilities exist, we omit them here and with rare advanced exceptions recommend against their use. As far as we are concerned, to allocate, you must use: <code>x = arena-&gt;construct&lt;T&gt;(...)</code>, where <code>...</code> are constructor args to <code>T</code> (possibly none); it returns <code>Arena::Handle&lt;T&gt;</code> which is, simply, <code>shared_ptr&lt;T&gt;</code>. (Whether it's <code>boost::shared_ptr</code> or <code>std::shared_ptr</code> is formally unspecified; but they have equivalent semantics.)</p>
<p >To distinguish between it and other things, the returned thing from <code>.construct&lt;T&gt;()</code> &ndash; <code>x</code> above &ndash; is called a <b>first-class SHM handle</b> (or just <b>SHM handle</b> or <b>handle</b>). It <em>is</em> a <code>shared_ptr&lt;T&gt;</code>, but this <em>particular</em> shared-pointer has an important property that is not otherwise obvious given its apparent type. Namely it is fitted (invisibly) with a <em>custom deleter</em> supplied by Flow-IPC. Equally importantly, the <em>handle</em> is the entity that can be transmitted (<b>lent</b>) to another process. That is the main reason it is a <em>first class</em> item. Certainly <em>subordinate</em> allocations occur <em>under</em> that handle in the future &ndash; for example a <code>vector</code> allocating its buffer &ndash; but these are not themselves transmitted (lent) between processes. Only the <em>first-class handle</em> is.</p>
<hr  />
<dl class="section user"><dt>Garbage-collection of SHM-allocated objects</dt><dd>A key feature and convention is that a SHM-handle, as returned by <code>.construct&lt;&gt;()</code>, is such that the underlying RAM is deallocated automatically. (Goes without saying possibly but do not try to manually <code>delete x.get()</code> or save an <code>x.get()</code> beyond <code>x</code> lifetime, or anything like that. Once a <code>shared_ptr</code>, always a <code>shared_ptr</code>.) Deallocation means return at least for further allocations in the same arena. (Internally it <em>might</em> also lead to return of RAM for general OS use in some situations. That's a hairy optimization detail that depends on the SHM-provider. Generally SHM-jemalloc is into such things, while SHM-classic less so.)</dd></dl>
<dl class="section user"><dt></dt><dd>A very important point is that the auto-deallocation occurs <em>on a cross-process basis</em>. If one never transmits (lends) <code>x</code> via IPC, then it acts like any other <code>shared_ptr</code>: ref-count-zero is reached; so the resource is returned (albeit to SHM-arena as opposed to the general heap). If one transmits it via IPC N (N &gt;= 1) times, however, then: The underlying data structure is deallocated once <em>all</em> first-class handles have reached ref-count-zero in their respective processes. That includes:<ul>
<li>The <code>shared_ptr</code> group of the original <code>x = ....construct&lt;T&gt;(...)</code> call.</li>
<li>For each <code>x_borrowed = ....borrow&lt;T&gt;(...)</code> upon receipt of <code>x</code> by a process over IPC: The <code>shared_ptr</code> group of that <code>x_borrowed</code>.</li>
</ul>
</dd></dl>
<dl class="section user"><dt></dt><dd>Perhaps in plainer English (?): The original constructed handle <code>shared_ptr</code>, plus each similarly-functioning <code>shared_ptr</code> obtained by a receiving process upon IPC transmission of the original, together form a multi-process meta-shared-pointer group; and the underlying memory is deallocated no earlier than that entire meta-shared-pointer group's ref-count reaches zero. (To-do: a diagram would help here.)</dd></dl>
<hr  />
<p >A super-important topic is what <code>T</code> can be. Without any complications, it can at least be a pointer-free plain-old data-type (POD) of arbitrary (but known at compile-time) depth/complexity:</p><ul>
<li>Integers, floating-point numbers, Booleans.</li>
<li><code>struct</code>, <code>union</code>, <code>class</code> aggregating any of the above (including themselves &ndash; meaning multi-level <code>struct</code>s et al).</li>
<li>Native, fixed-size arrays collecting any of the above (including themselves).</li>
</ul>
<dl class="section note"><dt>Note</dt><dd>An <code>array&lt;..., N&gt;</code> is allowed in a POD: it is a <code>class</code> or <code>struct</code> containing a native fixed-size array of <code>...</code>s. <code>N</code> is a constant known at compile time. Informally we recommend the use of <code>array&lt;T, N&gt; x</code> over <code>T x[N]</code>: there is literally no downside (including perf).</dd>
<dd>
Raw pointers are not allowed in a POD in this context (nor are they, generally, allowed in the larger context of what can go into SHM-stored <code>T</code>).</dd></dl>
<p>That's pretty good. Yet it is not good enough for a huge-range of algorithms. Ultimately that's because a POD can't have pointers (or another way of saying it: dynamically-sized arrays). Support for pointers/dynamically-sized arrays is required, among other things, to be able to store STL-compliant containers. For that matter intrusive/manual data structures (such as explicit linked lists or trees) also require pointers. Fortunately:</p>
<h3>Allocating non-PODs (data structures with pointers, dynamically-sized items, and/or STL-compliant containers)</h3>
<p >Flow-IPC provides extensive support for SHM-stored pointers and, by extension, STL-compliant containers (and intrusive/manual data structures).</p>
<p >At its formal core, in order for such a <code>T</code> (one involving pointers) to be <code>.construct&lt;T&gt;()</code>ible, a pointer field <code>m_x</code> inside (directly or indirectly via more such pointers) <code>T</code> must be as follows. Suppose the pointee's type is <code>P</code>.</p><ul>
<li>Its type must not be <code>P*</code> (raw pointer) but rather: <code>Arena::Pointer&lt;P&gt;</code>, where <code>Arena::construct&lt;T&gt;()</code> was the method used to construct.<ul>
<li><code>Pointer&lt;P&gt;</code> is called a <b>fancy pointer</b> (yes, actual technical name in C++-world; no, it is not the same as smart pointer) to <code>P</code>. (For the curious: With SHM-classic it is <code>boost::interprocess::offset_ptr&lt;P&gt;</code>. With SHM-jemalloc it is a custom type written by us, internally storing a SHM pool ID and offset within the IDed pool.)</li>
</ul>
</li>
<li><code>m_p</code> (of type <code>Pointer&lt;P&gt;</code>) must have been obtained as follows: <code>m_p = Pointer&lt;P&gt;(static_cast&lt;P*&gt;(arena.allocate(sizeof(P)))</code>.</li>
<li>If <code>T::~T()</code> is invoked, it must ensure that the following occurs: <code>arena.deallocate(static_cast&lt;void*&gt;(m_p.get()))</code>.</li>
</ul>
<p >Indeed, if your plan is to store actual pointer-like fields in the data structure rooted in <code>T</code> (context: <code>.construct&lt;T&gt;()</code>) &ndash; meaning <code>T</code> is an intrusive/manual data structure &ndash; then you must ensure all that somewhat-hairy stuff. Whereas when working with the regular heap you'd just:</p><ul>
<li>Use <code>P* m_p</code>. No fancy-pointer needed.</li>
<li><code>m_p = new P</code> to allocate.</li>
<li>If <code>T::~T()</code> is invoked: <code>delete m_p</code>.</li>
</ul>
<p >So that would definitely be taxing to code. And indeed, particularly with legacy code, such measures may be necessary. We recommend against it whenever possible. Instead use <b>STL-compliant SHM-friendly containers</b>. These include: <code>boost::container::*</code> (<code>basic_string</code>, <code>list</code>, <code>vector</code>, <code>map</code>, <code>deque</code>, etc.), <code>boost::unordered_*</code> (<code>map</code>, <code>set</code>, etc.), and <code>flow::util::Basic_blob</code> (and <code>Blob</code> et al).</p>
<dl class="section warning"><dt>Warning</dt><dd>As a rule <code>std::</code> containers are not SHM-friendly, at least in gcc as of gcc-9; they assume raw-pointer-using allocators. By contrast <code>boost::container::*</code> corrected those issues. <code>std::vector</code> happens to be okay in gcc-8 at least, but it's safer to just go with <code>boost::container</code>. E.g., its <code>std::list</code> bro is broken in this regard.</dd></dl>
<p>Once you've chosen your STL-compliant SHM-friendly container type &ndash; or developed your own! &ndash; you must take care to, also, specify (as the <code><a class="el" href="classAllocator.html">Allocator</a></code> template paramater to the container type) the <b>SHM-allocating allocator</b> we've provided. (boost.interprocess provides an allocator template for similar use; but it is stateful, which is a huge pain in the butt &ndash; plus it uses additional RAM to store the allocator pointer.) Use this allocator:</p>
<div class="fragment"><div class="line"><span class="keyword">template</span>&lt;<span class="keyword">typename</span> T&gt;</div>
<div class="line"><span class="keyword">using </span>Shm_allocator = Session::Allocator&lt;T&gt;;</div>
<div class="line"> </div>
<div class="line"><span class="comment">// Alternatively:</span></div>
<div class="line"><span class="keyword">using </span>Arena = Session::Arena;</div>
<div class="line"><span class="keyword">template</span>&lt;<span class="keyword">typename</span> T&gt;</div>
<div class="line"><span class="keyword">using </span>Shm_allocator = <a class="code hl_class" href="classipc_1_1shm_1_1stl_1_1Stateless__allocator.html">ipc::shm::stl::Stateless_allocator&lt;T, Arena&gt;</a>;</div>
<div class="ttc" id="aclassipc_1_1shm_1_1stl_1_1Stateless__allocator_html"><div class="ttname"><a href="classipc_1_1shm_1_1stl_1_1Stateless__allocator.html">ipc::shm::stl::Stateless_allocator</a></div><div class="ttdoc">Stateless allocator usable with STL-compliant containers to store (or merely read) them directly in S...</div><div class="ttdef"><b>Definition:</b> <a href="stateless__allocator_8hpp_source.html#l00118">stateless_allocator.hpp:119</a></div></div>
</div><!-- fragment --><p >For example here's an aggregate <code>T</code> consisting of a few things, including scalars and containers, being constructed:</p>
<div class="fragment"><div class="line"><span class="keyword">struct </span>Widget</div>
<div class="line">{</div>
<div class="line">  <span class="keyword">struct </span>Node</div>
<div class="line">  {</div>
<div class="line">    <span class="keywordtype">int</span> m_int;</div>
<div class="line">    boost::container::vector&lt;float, Shm_allocator&lt;float&gt;&gt; m_float_vec;</div>
<div class="line">  };</div>
<div class="line">  <span class="keyword">using </span>String = boost::container::basic_string&lt;char, std::char_traits&lt;char&gt;, Shm_allocator&lt;char&gt;&gt;;</div>
<div class="line"> </div>
<div class="line">  <span class="keywordtype">bool</span> m_flag;</div>
<div class="line">  String m_str;</div>
<div class="line">  boost::unordered_map&lt;String, Node, std::hash&lt;String&gt;, std::equal_to&lt;String&gt;,</div>
<div class="line">                       Shm_allocator&lt;std::pair&lt;const String, Node&gt;&gt;</div>
<div class="line">    m_str_to_node_map;</div>
<div class="line">};</div>
<div class="line"><span class="keyword">auto</span> x = session.session_shm()-&gt;construct&lt;Widget&gt;(); <span class="comment">// This uses a default constructor, but generally one can (carefully, when initializing) use args.</span></div>
<div class="line">x-&gt;m_flag = <span class="keyword">true</span>;</div>
<div class="line"><span class="comment">// ...</span></div>
<div class="line"> </div>
<div class="line"><span class="comment">// Here is what it would have looked like without SHM support (for comparison):</span></div>
<div class="line"><span class="keyword">struct </span>Widget</div>
<div class="line">{</div>
<div class="line">  <span class="keyword">struct </span>Node</div>
<div class="line">  {</div>
<div class="line">    <span class="keywordtype">int</span> m_int;</div>
<div class="line">    std::vector&lt;float&gt; m_float_vec;</div>
<div class="line">  };</div>
<div class="line"> </div>
<div class="line">  <span class="keywordtype">bool</span> m_flag;</div>
<div class="line">  std::string m_str;</div>
<div class="line">  boost::unordered_map&lt;std::string, Node&gt; m_str_to_node_map;</div>
<div class="line">};</div>
<div class="line"><span class="keyword">auto</span> x = <span class="keyword">new</span> Widget;</div>
<div class="line">x-&gt;m_flag = <span class="keyword">true</span>;</div>
<div class="line"><span class="comment">// ...</span></div>
</div><!-- fragment --><p >As you can see the power is there; one just needs to remember to keep supplying the proper allocator and SHM-friendly container templates at all levels. The code is not <em>exactly</em> the same, but with some convenience aliases it's quite similar.</p>
<hr  />
<dl class="section user"><dt>Stack (et al) use</dt><dd>If you intend to transmit (lend) a <code>T</code>-typed object via IPC, then of course you must <code>construct&lt;T&gt;()</code> it as shown above.</dd></dl>
<dl class="section user"><dt></dt><dd>However a <code>T</code> can also be used for other purposes and yet still be held partially in SHM. For example you might build up a <code>T = Widget::String</code> from the example above and then <code>move()</code> or copy it onto <code>x-&gt;m_str</code>, where <code>x = arena.construct&lt;Widget&gt;()</code>, such that you intend to actually IPC-transmit first-class SHM-handle <code>x</code>. In that case you can still <code>construct&lt;T&gt;()</code> the intermediate <code>Widget::String</code> &ndash; no problem &ndash; and use it in whatever way you need, even if you aren't going to transmit that guy but just use it locally and then let it be deallocated.</dd></dl>
<dl class="section user"><dt></dt><dd>However in such situations it would make your code less verbose and potentially faster &ndash; by avoiding unnecessary SHM use &ndash; to place the <code>T</code> on the stack (among other things; could be heap too). E.g.: <code>x = session.session_shm()-&gt;construct&lt;Widget&gt;(); Widget::String temp_str; ...mutate temp_str...; x-&gt;m_str = std::move(temp_str);</code>. Note <code>temp_str</code> did not need to be itself <code>construct&lt;&gt;()</code> ed. The <code>String</code> <em>itself</em> (the outer fixed-size thing of size <code>sizeof(String)</code>) is on the stack; but whatever it needed to actually allocate on its own behalf it would have used SHM for (due to the allocator properly being configured as part of the <code>String</code> alias).</dd></dl>
<dl class="section user"><dt></dt><dd>If you are familiar with C++ allocators, then this is perhaps no surprise.</dd></dl>
<hr  />
<h3>Mutating a SHM-stored <code>T</code></h3>
<p >So you've constructed it. How do you fill it out further? Well, if <code>T</code> is a POD as defined above, then you just do it; no different from the usual. E.g. here we modify the POD-ish parts of a <code>Widget</code>:</p>
<div class="fragment"><div class="line">x-&gt;m_flag = <span class="keyword">true</span>;</div>
<div class="line">x-&gt;m_str[0] = <span class="charliteral">&#39;X&#39;</span>; <span class="comment">// Assumes, of course, that by this point: x-&gt;m_str.size() &gt; 0.  Else this would be undefined behavior/buffer overflow.</span></div>
</div><!-- fragment --><p >If, however, what you're doing will require some part of <code>T</code> to <em>allocate</em> on its behalf, then it'll need to use its allocator. But wait... why won't that just work? After all we've specified the <code><a class="el" href="classAllocator.html">Allocator</a></code> template param so nicely in our <code>Widget</code> declaration! Answer: We've declared the allocator <em>type</em> to use, yes, and that is critical. However, the container code needs to know which arena to actually allocate-in. <code>arena-&gt;construct&lt;T&gt;()</code> is Flow-IPC code, and it knows it's being invoked on <code>*arena</code>, so it does (internally) what's needed. However consider this:</p>
<div class="fragment"><div class="line">assert(x-&gt;m_str.empty());</div>
<div class="line">x-&gt;m_str.resize(128); <span class="comment">// This should make x-&gt;m_str contain 128 NUL characters.</span></div>
</div><!-- fragment --><p ><code>basic_string::resize()</code>, internally, will invoke its "stored" allocator of type <code>Shm_allocator</code> to allocate at least a 128-buffer. But <code>Shm_allocator = Stateless_allocator&lt;...&gt;</code> is a <b>stateless allocator</b>. That means the allocator object "stored" inside the container outer structure has size 0: it takes no space and has no state. So the container code can't "know" from what <code>Arena</code> to allocate space!</p>
<p >To make allocating mutators (in this case <code>.resize()</code>) work you must <b>activate the arena</b>. This is done in thread-local fashion by using a RAII-style object, an <a class="el" href="classipc_1_1shm_1_1stl_1_1Arena__activator.html" title="RAII-style class operating a stack-like notion of a the given thread&#39;s currently active SHM-aware Are...">ipc::shm::stl::Arena_activator</a>:</p>
<div class="fragment"><div class="line"><span class="keyword">using </span>Activator = <a class="code hl_class" href="classipc_1_1shm_1_1stl_1_1Arena__activator.html">ipc::shm::stl::Arena_activator&lt;Session::Arena&gt;</a>;</div>
<div class="line"> </div>
<div class="line"><span class="keyword">auto</span> x = session.session_shm()-&gt;construct&lt;Widget&gt;();</div>
<div class="line"><span class="keyword">auto</span> y = session.app_shm()-&gt;construct&lt;Widget&gt;();</div>
<div class="line"> </div>
<div class="line">{</div>
<div class="line">  Activator ctx_sess(session.session_shm()); <span class="comment">// When modifying Shm_allocator-using things best to do this at the top.</span></div>
<div class="line"> </div>
<div class="line">  x-&gt;m_flag = <span class="keyword">true</span>; <span class="comment">// ctx_sess is active but has zero effect on this (harmless and zero perf impact).</span></div>
<div class="line">  x-&gt;m_str.resize(128); <span class="comment">// ctx_sess is active and will cause proper arena to be used for allocation.</span></div>
<div class="line"> </div>
<div class="line">  {</div>
<div class="line">    Activator ctx_app(session.app_shm()); <span class="comment">// Activators &quot;stack&quot;: the latest one to be cted and not dted is active.</span></div>
<div class="line"> </div>
<div class="line">    y-&gt;m_str.resize(128); <span class="comment">// ctx_app is active and will cause proper arena to be used for allocation.</span></div>
<div class="line">  }</div>
<div class="line">  <span class="comment">// ctx_sess is active again.</span></div>
<div class="line"> </div>
<div class="line">  x-&gt;m_str.resize(1024);</div>
<div class="line">}</div>
<div class="line"> </div>
<div class="line"><span class="comment">// If no Activator is active here (in this thread) this would trip internal assert() inside Shm_allocator.</span></div>
<div class="line">x-&gt;m_str.resize(128);</div>
<div class="ttc" id="aclassipc_1_1shm_1_1stl_1_1Arena__activator_html"><div class="ttname"><a href="classipc_1_1shm_1_1stl_1_1Arena__activator.html">ipc::shm::stl::Arena_activator</a></div><div class="ttdoc">RAII-style class operating a stack-like notion of a the given thread's currently active SHM-aware Are...</div><div class="ttdef"><b>Definition:</b> <a href="arena__activator_8hpp_source.html#l00040">arena_activator.hpp:41</a></div></div>
</div><!-- fragment --><p >Note well: <code>Arena_activator</code>s stack as shown. So you can work with multiple arenas in close proximity. The key point is it affects only the current thread.</p>
<dl class="section note"><dt>Note</dt><dd>boost.interprocess provides a <em>stateful</em> allocator. Using those is quite painful, especially when using containers of containers, but let's not get into it here. Our syntax is much less painful, though there's no free lunch: we have to use the activator statement to make it work, and there is some compute used internally to those to make the thread-local state apply. We feel this trade-off is well worth it in our favor.</dd></dl>
<hr  />
<dl class="section user"><dt>When MUST/SHOULD you activate an arena?</dt><dd>We've answered this already: whenever you're modifying a data structure living in SHM, such that it is of a type fitted with <code><a class="el" href="classAllocator.html">Allocator</a></code> template parameter equal to <code>Stateless_allocator&lt;something&gt;</code>, and that operation might want to allocate or deallocate (via its <code><a class="el" href="classAllocator.html">Allocator</a></code>). In typical use of STL-compliant containers we don't usually have to think about such technicalities; most people don't really know or care about STL-allocators at all. Hence knowing when specifically you <em>must</em> do it is arguably a somewhat tall order. Possibly.</dd></dl>
<dl class="section user"><dt></dt><dd>So, informally, a better question might be: what are convenient rules of thumb that'll ensure you do the right thing, without worrying too much about allocator technicalities? We suggest these:<ul>
<li>If your code block is <code>const</code> in nature &ndash; you're not modifying your <code>T</code> at all &ndash; then it is <em>never</em> necessary. (In point of fact, with SHM-jemalloc not only is it not necessary, it is also <em>impossible</em> on the receiving side of an IPC-transmission of a SHM-handle. There <em>is</em> simply no arena to activate in that situation. And as of this writing, writing on the receiving side is disallowed at the kernel level anyway. With SHM-classic, though, both allocation and writing <em>is</em> possible. Nevertheless even then: if your receiving-side algorithm is read-only w/r/t a SHM-stored <code>T</code>, then you need not &ndash; and should not &ndash; activate any arena.)</li>
<li>If your code block is non-<code>const</code> in nature &ndash; you are making <em>some</em> modifications to <code>T</code> within it:<ul>
<li>Generally you should just activate the arena at the top of the code block: better safe than sorry. Those statements that don't touch an allocator won't be affected; but those that do will just work nicely. And you needn't worry about the details of which is which, if it's active at the top.</li>
<li>However, if you're extremely perf-conscious, and some code paths don't touch anything allocator-related, then you could be somewhat conditional about it, activating the arena in smaller sub-blocks of your overall non-<code>const</code> block.</li>
</ul>
</li>
</ul>
</dd></dl>
<dl class="section user"><dt></dt><dd>As a reminder, it is not necessary to activate an arena when <code>.construct&lt;T&gt;()</code>ing the outer object (first-class SHM-handle). <code>.construct&lt;T&gt;()</code> will take care of it.</dd></dl>
<dl class="section user"><dt></dt><dd>It is also emphatically <em>not</em> necessary to activate an arena when the outer object might get destroyed (due to the SHM-handle reaching ref-count zero). Our internal custom-deleter code will take care of it.</dd></dl>
<hr  />
<h3>Transmission of SHM-stored data structures: Lend and borrow</h3>
<p >We've made references to transmission/lending a few times already. E.g., we've mentioned a <code>construct()</code>ed <code>T</code> gets deallocated once all handles &ndash; the original and any <em>borrowed</em> ones &ndash; have reached ref-count zero. Time to fill that big gap. How would those other handles spring into existence, presumably in other processes?</p>
<p >It's all well and good to construct and modify a thing in SHM, but it's not useful until another process receives it via IPC and begins their own work, whether read-only or read-write. Until then you might as well have just constructed in the regular heap in the first place.</p>
<p >These are the steps in sharing a <b>first-class SHM-handle</b> <code>x = some_arena.construct&lt;T&gt;(...)</code>. Note that only a first-class handle can be shared: you can't share "just" <code>x-&gt;m_str</code> or something. And recall that <code>x</code> is <code>shared_ptr&lt;T&gt;</code> (which is also aliased from <code>Arena::Handle&lt;T&gt;</code> for stylistic purposes). The original creator of the handle &ndash; the guy to call <code>construct()</code> &ndash; is called the handle's <b>owner</b> or <b>owner process</b>. The owner can transmit a handle to another process, called the <b>borrower (process)</b>. When doing so the owner is called the <b>lender (process)</b>.</p><ul>
<li>Lender process (has <code>x</code>): It prepares <code>x</code> for transmission by calling: <code>const auto lend_blob = session.lend_object(x)</code>. <code>lend_blob</code> is a <em>small</em> encoding of certain information (a few bytes). It needs to be copied into/out of a transport: which is far superior to doing that to the entirety of the actual object!</li>
<li>Lender process + borrower process in tandem: Transmit <code>lend_blob</code> to the former from the latter, via any IPC technique whatsoever.</li>
<li>Borrower process (wants its own <code>x_borrowed</code>, like <code>x</code>): It obtains <code>x_borrowed</code> by calling: <code>auto x_borrowed = session.borrow_object&lt;T&gt;(lend_blob)</code>.</li>
</ul>
<p >Et voilà! You've got yourself a guy just like <code>x</code> but in another process. <code>x_borrowed</code> is, also, a <code>shared_ptr&lt;T&gt;</code>. One thing to note here is that the <em>arena</em> is <em>not</em> a part of this procedure. The session is; and the session in and of itself determines who's the recipient. Another thing to note is that the borrower code of yours must know the type <code>T</code>; if it does not match then behavior is undefined.</p>
<dl class="section note"><dt>Note</dt><dd>You can alternatively use similar <code>.lend_object()</code> and <code>.borrow_object()</code> method on SHM-arena and SHM-session objects provided by <a class="el" href="namespaceipc_1_1shm.html" title="Modules for SHared Memory (SHM) support.">ipc::shm</a>. However this opens up various subtleties beyond our scope here. The Reference (in particular at least <a class="el" href="classipc_1_1session_1_1shm_1_1classic_1_1Session__mv.html#a07c9b1803567c2714ae48433e756047f">here</a>, <a class="el" href="classipc_1_1session_1_1shm_1_1arena__lend_1_1jemalloc_1_1Session__mv.html#ae30ae4b65cd4178fa26eed660688829f">here</a>, and <a class="el" href="classipc_1_1session_1_1shm_1_1arena__lend_1_1jemalloc_1_1Session__mv.html#ad63c175a57f314c1ed3d4a78329030cd">here</a>) will help shed light. This technique may be useful in particular if working with arenas outside the <a class="el" href="namespaceipc_1_1session.html" title="Flow-IPC module providing the broad lifecycle and shared-resource organization – via the session conc...">ipc::session</a> paradigm.</dd></dl>
<p>Note that we wrote the above in terms of <em>lender</em> and borrower; not specifically <em>owner</em> and borrower. That is because a <em>borrower</em> can act as a <em>lender</em> and transmit the handle to yet another process. This is called <b>proxying</b>. However, in its current version, SHM-jemalloc does not support proxying (a future version likely will, as the feature was designed from the start). SHM-classic does fully support proxying. That said, within an <code><a class="el" href="namespaceipc_1_1session.html" title="Flow-IPC module providing the broad lifecycle and shared-resource organization – via the session conc...">ipc::session</a></code>-based IPC universe proxying is somewhat unlikely. We'll cut that discussion off here, as it gets into hairy super-advanced topics.</p>
<hr  />
<dl class="section user"><dt>What does it mean for the owner and borrower types to match?</dt><dd>We glossed over it; we simply said the owner <code>T</code> and the borrower-<code>T</code> (template param to <code>borrow_object()</code> and therefore to <code>shared_ptr</code> for <code>x_borrowed</code>) must "match." What does that mean? Does it mean they must be the same type/bit-compatible?</dd></dl>
<dl class="section user"><dt></dt><dd>Yes and no. Depends. First if <code>T</code> is a POD (as defined earlier in this Manual page), then yes: <code>T</code> should just be the same type, and that's that. Now let's say <code>T</code> uses STL-compliant members (at any depth) and/or pointers (ditto). Then: The short answer depends on whether your code is generically meant to work with SHM-classic or SHM-jemalloc interchangeably &ndash; or only one of them specifically.<ul>
<li>If it should work generically with either:<ul>
<li>Owner-side <code>T</code> must (as we've explained) use <code>Session::Allocator</code>-equipped types and/or <code>Session::Allocator::Pointer</code>-typed pointers. By contrast borrower-side <code>T</code> must be an identical type <em>except</em> the allocator type for both of those things within <code>T</code> shall be <em>not</em> <code>Session::Allocator</code> but rather: <code>Session::Borrower_allocator</code>.</li>
</ul>
</li>
<li>If it intends to specifically work with (and use specific features/properties of) SHM-classic:<ul>
<li><code>T</code> can be identical on both sides. In any case <a class="el" href="classipc_1_1session_1_1shm_1_1classic_1_1Session__mv.html#aecfab9c96933a832be7d2ad91be33532" title="Equals Allocator; provided for generic programming for algorithms that would use classic::Session_mv ...">ipc::session::shm::classic::Session_mv::Borrower_allocator</a> just aliases to <code>"...::Allocator"</code>.</li>
</ul>
</li>
<li>If it intends to specifically work with (and use specific features/properties of) SHM-jemalloc: Use <code>Session::Borrower_allocator</code>.</li>
</ul>
</dd></dl>
<dl class="section user"><dt></dt><dd>To recap: if it's a POD, same <code>T</code> on all sides. If it uses STL-compliant/pointer stuff, then the most generic way it to use <code>Session::Allocator</code> in owner code; <code>Session::Borrower_allocator</code> in borrower code. And if targeting SHM-classic specifically, it is <em>okay</em> &ndash; for conciseness though not genericness &ndash; to just use the same type <code>T</code> on both sides, period.</dd></dl>
<dl class="section user"><dt></dt><dd>What's going on here, you ask? We'd rather not get into it here; we've provided the recipe. But various docs inside <a class="el" href="namespaceipc_1_1shm.html" title="Modules for SHared Memory (SHM) support.">ipc::shm</a> explain all the subleties. Long story short: SHM-classic is highly symmetric and (relatively) simple, so the borrower and owner are really internally operating on the same SHM-pool. In SHM-jemalloc only the owner even <em>has</em> the arena per se; the borrower has only a read-only view into parts of it &ndash; so it needs a special, degenerate "borrower" allocator which is used not to <em>allocate</em> but to only interpret pointers properly. (Whereas on the owner side it is used for that <em>and</em> allocation code.)</dd></dl>
<hr  />
<p >To close the loop: how to, in fact, IPC-transmit <code>lend_blob</code> &ndash; the thing returned by <code>lend_object()</code> and fed to <code>borrow_object()</code>? Well, it's just a little blob, so you can do it however you want. However, if you're using <code>struc::Channel</code> (<a class="el" href="chan_struct.html">Structured Message Transport</a>) &ndash; or for some odd reason capnp but without <code>struc::Channel</code> &ndash; then we've made a couple of utilities to reduce your boiler-plate. Here's how to use it:</p>
<p >Example schema:</p>
<div class="fragment"><div class="line">@0xa780a4869d13f307;</div>
<div class="line">using Cxx = import &quot;/capnp/c++.capnp&quot;;</div>
<div class="line">using Common = import &quot;/ipc/transport/struc/shm/schema/common.capnp&quot;; # Flow-IPC supplies this.</div>
<div class="line"> </div>
<div class="line">using ShmHandle = Common.ShmHandle; # It&#39;s just a blob holder really.</div>
<div class="line"> </div>
<div class="line">$Cxx.namespace(&quot;my_meta_app::capnp&quot;);</div>
<div class="line"> </div>
<div class="line"># ...</div>
<div class="line"> </div>
<div class="line">struct SomeMsg</div>
<div class="line">{</div>
<div class="line">  # Example of message struct or sub-struct that among possible other things conveys a first-class SHM-handle.</div>
<div class="line"> </div>
<div class="line">  # ...</div>
<div class="line"> </div>
<div class="line">  widgetHandle @7 :ShmHandle;</div>
<div class="line"> </div>
<div class="line">  # ...</div>
<div class="line">}</div>
<div class="line"> </div>
<div class="line"># ...</div>
</div><!-- fragment --><p >Example owner/lender code:</p>
<div class="fragment"><div class="line"><span class="keyword">auto</span> x = session.session_shm()-&gt;construct&lt;Widget&gt;();</div>
<div class="line"><span class="comment">// ...Fill out *x....</span></div>
<div class="line"> </div>
<div class="line"><span class="keyword">auto</span> msg = cool_channel.create_msg();</div>
<div class="line"><span class="comment">// Ready the lend_blob-storage inside the out-message.</span></div>
<div class="line"><span class="keyword">auto</span> widget_handle_root = msg.body_root()-&gt;initSomeMsg().initWidgetHandle();</div>
<div class="line"><span class="comment">// Perform the lend_object() step and load the result into the out-message.</span></div>
<div class="line"><a class="code hl_function" href="namespaceipc_1_1transport_1_1struc_1_1shm.html#ab592ce6eddbe24c57dc71f34018fb042">ipc::transport::struc::shm::capnp_set_lent_shm_handle</a>(&amp;widget_handle_root, <span class="comment">// Output.</span></div>
<div class="line">                                                      session.lend_object(x)); <span class="comment">// Input.</span></div>
<div class="line"> </div>
<div class="line"><span class="comment">// IPC-transmit it via struc::Channel.</span></div>
<div class="line">cool_channel.send(msg);</div>
<div class="ttc" id="anamespaceipc_1_1transport_1_1struc_1_1shm_html_ab592ce6eddbe24c57dc71f34018fb042"><div class="ttname"><a href="namespaceipc_1_1transport_1_1struc_1_1shm.html#ab592ce6eddbe24c57dc71f34018fb042">ipc::transport::struc::shm::capnp_set_lent_shm_handle</a></div><div class="ttdeci">void capnp_set_lent_shm_handle(schema::ShmHandle::Builder *shm_handle_root, const flow::util::Blob_sans_log_context &amp;lend_result)</div><div class="ttdoc">Utility that saves the result of a Shm_session1::lend_object&lt;T&gt;(const shared_ptr&lt;T&gt;&amp;) result into the...</div><div class="ttdef"><b>Definition:</b> <a href="ipc__shm_2src_2ipc_2transport_2struc_2shm_2util_8cpp_source.html#l00028">util.cpp:28</a></div></div>
</div><!-- fragment --><p >And counterpart borrower code:</p>
<div class="fragment"><div class="line">flow::util::Blob_sans_log_context lend_blob;</div>
<div class="line"><a class="code hl_function" href="namespaceipc_1_1transport_1_1struc_1_1shm.html#adb38fe88cfd5e758ef52c482d2902669">ipc::transport::struc::shm::capnp_get_shm_handle_to_borrow</a>(msg-&gt;body_root().getSomeMsg().getWidgetHandle(), &amp;lend_blob);</div>
<div class="line"> </div>
<div class="line"><span class="keyword">auto</span> x_borrowed = session.borrow_object&lt;Widget_brw&gt;(lend_blob);</div>
<div class="line">FLOW_LOG_INFO(<span class="stringliteral">&quot;Hey, let&#39;s read inside SHM after receiving SHM-handle: [&quot;</span> &lt;&lt; x_borrowed-&gt;m_flag &lt;&lt; <span class="stringliteral">&quot;].&quot;</span>);</div>
<div class="ttc" id="anamespaceipc_1_1transport_1_1struc_1_1shm_html_adb38fe88cfd5e758ef52c482d2902669"><div class="ttname"><a href="namespaceipc_1_1transport_1_1struc_1_1shm.html#adb38fe88cfd5e758ef52c482d2902669">ipc::transport::struc::shm::capnp_get_shm_handle_to_borrow</a></div><div class="ttdeci">void capnp_get_shm_handle_to_borrow(const schema::ShmHandle::Reader &amp;shm_handle_root, flow::util::Blob_sans_log_context *arg_to_borrow)</div><div class="ttdoc">Utility that's the reverse of capnp_set_lent_shm_handle() to be invoked on the deserializing side.</div><div class="ttdef"><b>Definition:</b> <a href="ipc__shm_2src_2ipc_2transport_2struc_2shm_2util_8cpp_source.html#l00050">util.cpp:50</a></div></div>
</div><!-- fragment --><p >Simple! That said we opportunistically note: The borrower-side is using <code>Widget_brw</code>, a type we have not explicitly provided the code for in the actual example. Per the side-bar above, with SHM-classic it could just be <code>Widget</code> (same as in owner); but with SHM-jemalloc and generically you'd need to define a mirror of <code>Widget</code> called <code>Widget_brw</code> which would use <code>Session::Borrower_allocator</code> instead of <code>Session::Allocator</code> all-over. We'll leave that as an exercise to the reader. Tip: You do <em>not</em> need to copy paste the same type twice. Use template trickery &ndash; perhaps <code>std::conditional_t</code> &ndash; to conveniently pick between the two <code>*llocator</code> templates depending on a compile-time <code>bool S_OWN_ELSE_BRW</code> template parameter perhaps.</p>
<h3>What can you do with a borrowed (received) data structured in SHM?</h3>
<p >The answer to this is sprinkled throughout the above. Nevertheless it seemed prudent to put a fine point on the answer as well as perhaps provide some algorithmic tips.</p>
<p >Firstly, in every case, you can access it in read-only fashion. The syntax is just C++ syntax.</p>
<p >Secondly, with SHM-classic specifically, you can do anything else including modifying the data structure which in turn includes operations that would further allocate in SHM.</p>
<dl class="section note"><dt>Note</dt><dd>The fact you <em>can</em> doesn't mean you <em>should</em>. Not that we're saying you shouldn't either. In any case consider you may need to synchronize access, if (upon receipt of a handle) at least one side will be writing, while another side would be reading. If your synchronization scheme isn't based on an IPC messaging protocol (e.g., "you write, then send something, then I write, then I send something, then you write..."), then you might need a mutex and/or condition variables. In this context you'd simply place a regular mutex and/or condition variable into SHM and have each side lock on it. Formally boost.interprocess provides <code>boost::interprocess::interprocess_mutex</code> (and a recursive variant) and <code>boost::interprocess::interprocess_condition</code> (et al). Informally these appear to use the same stuff as boost.thread <code>mutex</code>, <code>condition_variable</code> (namely in POSIX they use pthread primitives), so you could probably just use those.</dd>
<dd>
So... even if you can (which you <em>can</em> with SHM-classic): Should you (1) have both sides write to an in-SHM structure; or at least (2) have only 1 side write but synchronize using an in-SHM mutex? Answer: Firstly realize that (2) is really (1) in disguise: Locking a <code>mutex</code> (et al) <em>writes</em> to it in its memory location... so while your algorithm may conceptually avoid writing from 1 of the 2 sides, it is actually still writing which means safety worries still apply: Could crash during a lock... and other stuff. So that leaves basically (1). The answer is: possibly. By default, all else being equal, it is best avoided for safety reasons (detailed in <a class="el" href="safety_perms.html">Safety and Permissions</a>). That said tons of applications and use cases need not be so paranoid, and they very well might enjoy using an algorithm, wherein 2 threads separated by a process boundary collaborate in read/write fashion on a common data structure.</dd></dl>
<p>So that leaves 2 remaining situations:</p><ul>
<li>You're writing code for SHM-jemalloc specifically.</li>
<li>You're writing generic code that would work with either SHM-provider.</li>
</ul>
<p >Either way: Your code then <em>shall not write to a borrowed in-SHM data structure</em>: it must read only. If the <em>owner</em> does want to write post-transmission, it must ensure such access is synchronized with concurrent reads on the borrower side. <em>You may not use a mutex and/or condition variable in-SHM to arrange such synchronization.</em> (A mutex-lock operation is itself a write and must not be used.) Don't despair: as mentioned earlier one can arrange synchronization via other algorithmic means such as IPC-messaging.</p>
<p >Again: You're not giving away those abilities of SHM-classic by choosing SHM-jemalloc <em>for free</em>. You get goodies in return: safety goodies and allocation-perf goodies. See <a class="el" href="transport_shm.html#shm_choice">back here</a>.</p>
<p >The next page is: <a class="el" href="transport_core.html">Transport Core Layer: Transmitting Unstructured Data</a>.</p>
<hr  />
<center><b>MANUAL NAVIGATION:</b> <a class="el" href="chan_struct_advanced.html">Preceding Page</a> - <a class="el" href="transport_core.html">Next Page</a> - <a href="./pages.html"><b>Table of Contents</b></a> - <a class="el" href="namespaceipc.html"><b>Reference</b></a></center> </div></div><!-- contents -->
</div><!-- PageDoc -->
<!-- start footer part -->
<hr class="footer"/><address class="footer"><small>
Generated on Thu May 2 2024 23:56:38 for Flow-IPC by&#160;<a href="https://www.doxygen.org/index.html"><img class="footer" src="doxygen.svg" width="104" height="31" alt="doxygen"/></a> 1.9.4
</small></address>
</body>
</html>
